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EXECUTIVE  SUMMARY 


The  Limited  Production  (LP)  Precision  Runway  Monitor  (PRM)  is  designed  to  provide 
faster  and  more  accurate  surveillance  of  aircraft  on  instrument  landing  approaches. 
The  system  includes  high  resolution  displays  with  specific  blunder  alarms  to  enable 
an  air  traffic  controller  to  precisely  monitor  landing  aircraft.  The  LP  PRM  is  a 
procurement  of  five  systems.  The  design  of  the  LP  PRM  is  based  on  the  Upgrade  PRM 
currently  in  operation  at  Raleigh-Durham  International  Airport  in  North  Carolina. 

This  test  plan  establishes  the  framework  to  guide  and  direct  the  LP  PRM  Operational 
Test  and  Evaluation  (OT&E)  test  program.  Sufficient  detail  is  provided  in  this 
document  to  define  and  direct  the  next  lower  level  of  test  dociamentation  and  to 
allocate  resources  to  the  test  program.  Specific  information  is  provided  on  the 
Critical  Operational  Issues  (COI)  to  be  addressed,  the  test  concept,  the  evaluation 
approach,  test  descriptions,  completion  criteria,  and  support  requirements. 
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1. 


INTRODUCTION. 


1 . 1  BACKGROUND . 

The  Precision  Runway  Monitor  (PRM)  Program  is  part  of  the  Federal  Aviation 
Administration  (FAA)  approach  to  improve  airport  capacity.  The  implementation  of 
the  PRM  system  will  achieve  significant  capacity  gains  at  airports  with  closely 
spaced  parallel  runways.  The  use  of  the  PRM  will  allow  simultaneous,  independent 
Instrument  Flight  Rules  (IFR)  approaches  at  these  airports. 

The  Limited  Production  (LP)  Precision  Runway  Monitor  (PRM)  is  designed  to  provide 
faster  and  more  accurate  surveillance  of  aircraft  on  instrument  landing  approaches. 
The  system  uses  a  phased  array,  electronically  scanned  monopulse  beacon  radar  to 
achieve  target  update  rates  of  up  to  1.0  second  and  a  mean  azimuth  accuracy  of 
1.0  milliradian.  The  system  includes  high  resolution  displays  with  specific  blunder 
alarms  to  enable  a  monitor  controller  to  precisely  monitor  landing  aircraft. 

The  LP  PRM  is  a  procurement  of  five  systems  from  the  Allied  Signal  Corporation.  The 
design  of  the  LP  PRM  is  based  on  the  Upgrade  PRM  currently  in  operation  at  Raleigh- 
Durham  International  Airport  in  North  Carolina.  The  contract  provides  for  full 
system  requirement  verification.  That  test  effort  will  be  fully  monitored  by  the 
FAA  Technical  Center. 

1.2  PURPOSE  OF  THE  TEST  PLAN. 

The  purpose  of  the  LP  PRM  Operational  Test  and  Evaluation  (OT&E)  Test  Plan  is  to 
define  the  test  approach  and  concept,  required  planning,  and  coordination  necessary 
to  fully  test  the  LP  PRM  during  the  OT&E  phase  of  testing. 

This  test  plan  establishes  the  framework  to  direct  the  LP  PRM  OT&E  test  program. 
Detail  is  provided  in  appendix  A  to  define  and  direct  the  development  of  the  LP  PRM 
OT&E  test  procedures  and  to  permit  an  allocation  of  resources  to  the  test  program. 
This  test  plan  allocates  responsibilities  to  the  various  organizations  involved  in 
the  test  effort. 

This  test  program  will  ensure  that  the  LP  PRM  meets  the  requirements  as  defined  in 
the  LP  PRM  Master  Test  Plan.  Those  requirements  were  developed  from  the  Operational 
Requirements  supplied  by  ATR-100.  Additional  requirements  have  been  added  to  the 
test  effort  to  ensure  that  the  PRM/NAS  system  meets  the  suitability,  effectiveness, 
maintainability,  supportability,  and  security  objectives  as  defined  in  FAA  Order 
1810. 4B.  The  TVRTM  in  appendix  B  contains  both  the  Master  Test  Plan  Requirements  and 
the  additional  OT&E  requirements. 

1.3  SCOPE  OF  THE  TEST  PLAN. 

The  scope  of  this  test  effort  is  strictly  limited  to  the  OT&E  tests  to  be  performed 
by  ACW  on  the  first  article  LP  PRM  system.  Any  subsequent  PRM  procurements  will 
require  their  own  OT&E  Integration  and  OT&E  Operational  test  efforts. 

This  test  effort  does  not  include  demonstrations  for  commercial  airlines  or  the 
pilot  union  community.  This  test  effort  is  not  intended  to  revalidate  the  concept 
of  PRM  monitored  simultaneous  approaches.  This  test  program  is  concerned  with 
determining  whether  the  LP  PRM  system  (including  contractor  provided  training  and 
documentation)  is  effective  and  suitable  for  inclusion  into  the  National  Airspace 
System  (NAS)  . 

2.  REFERENCE  DOCUMENTATION. 

a.  Limited  Production  Precision  Runway  Monitor  (PRM)  Master  Test  Plan, 
November  1992,  DOT/FAA/CT-TN92/23 . 
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b.  Air  Traffic  Operational  Requirements  for  a  PRM  System,  1991. 

c.  Allied  Sigal,  Master  Test  Plan,  A001-01“002. 

d.  Allied  Signal,  Phase  1  Test  Plan,  A002-001-001 . 

e.  Allied  Signal,  Phase  2  Test  Plan,  BOOS-OQl-OOl . 

f.  Electronic  Scan  Precision  Runway  Monitor  (E^SCAN  PRM),  FAA--E-'2887  Rev.  A 
5  November  1993. 

g.  FAA  NAS  Test  and  Evaluation  Policy,  10/22/92,  FAA-ORDER--1810 . 4B . 

h.  Content  and  Format  Requirements  for  the  Preparation  of  Test  and 
Evaluation  Documentation,  FAA-STD-024B  August  22,  1994. 

3.  SYSTEM  DESCRIPTION. 

3.1  SYSTEM  OVERVIEW. 

The  LP  PRM  system  is  a  monopulse  secondary  surveillance  radar  system  that  detects, 
acquires,  tracks,  and  displays  beacon-equipped  air  traffic  within  a  predefined 
coverage  area.  Aircraft  data  is  presented  to  air  traffic  controllers  to  assist  in 
monitoring  and  maintaining  the  required  separation  of  aircraft  on  approaches  to 
closely  spaced  parallel  runways.  The  LP  PRM  system  is  comprised  of  five  main 
subsystems  as  listed  below: 

a.  Beacon  Radar  Subsystem  (BRS) 

b.  Radar  Display  Subsystem  (RDS) 

c.  Communication  Subsystem  (CS) 

d.  Confidence  and  Performance  Monitoring  Subsystem  (CPMS) 

e.  Recording  and  Playback  Subsystem  (RPS) 

The  LP  PRM  equipment  is  located  in  two  separate  facilities;  the  Transmit /Receive 
(T/R)  and  Operations  sites.  The  T/R  site  contains  the  BRS,  CPMS,  and  portions  of 
the  CS.  The  Operations  site  contains  RDS,  CPMS,  RPS,  and  portions  of  the  CS. 

Figure  3.1-1  PRM  System  Block  diagram  shows  a  functional  depiction  of  the  PR  system. 
The  BRS  provides  beacon  surveillance,  acquisition,  and  tracking.  The  BRS 
interrogates  aircraft  transponders,  processes  the  replies,  establishes  and  updates 
system  tracks,  and  transmits  track  data  to  the  RDS.  The  BRS  consists  of  an 
electronically  scanned  phased  array  antenna,  connected  through  a  control  transfer 
switch  to  redundant  channels.  Each  channel  consists  of  an  interrogator/receiver,  a 
signal  processor,  a  test  target  generator,  a  data  processor,  an  executive  monitor, 
and  associated  power  supplies. 

The  RDS  provides  the  display  of  the  selected  coverage  area  and  map  data, 
surveillance  data,  system  status  data,  and  system  alert  data.  The  RDS  receives 
target  track  data  from  the  BRS  (via  the  CS) ;  correlates  the  track  data  with  ARTS 
data;  presents  past,  current  and  predicted  track  data  on  high  resolution  color 
displays,  and  sends  data  to  the  RPS  for  recording.  The  RDS  also  generates  and 
presents  visual  and  aural  blunder  alerts.  The  RDS  consists  of  up  to  four  display 
consoles  and  associated  processors  as  well  as  two  ARTS  interface  units.  In  the 
normal  operational  configuration,  one  display  console  is  located  in  the  equipment 
room  at  the  operations  site  and  the  remaining  display  consoles  (one  for  each 
approach)  are  located  in  the  Air  Traffic  Terminal  Radar  Approach  Control  (TRACON) . 

The  CS  provides  both  inter-  and  intrasite  communications  connecting  all  major 
subsystem  together.  The  CS  is  a  fully  redundant  ethernet  local  area  network  (LAN) 
with  full  error  detection  and  automatic  switching. 
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The  CPMS  provides  for  the  monitoring  of  critical  system  performance  parameters  in 
the  BRS,  RDS,  RPS,  and  CS  as  well  as  automatic  BRS  channel  switching.  A  PRM  Status 
Display  (PSD)  provides  a  keyboard  interface  to  initiate  CPMS  maintenance  monitoring 
including:  BRS  system  testing,  subsystem  and  environmental  status  monitoring,  and 
fault  detection  and  isolation.  The  CPMS  consist  of  the  Executive  Monitors,  two 
PSDs,  and  a  speaker  for  alarm  generation. 

The  RPS  provides  for  the  recording  and  playback  of  operational  data.  The  RPS 
playback  and  listing  functions  allow  operational  data  to  be  played  back  on  the 
display  consoles  or  printed  onto  paper.  The  RPS  incorporates  two  redundant  9-track 
tape  recorder/playback  units  and  a  printer. 

The  critical  power  is  provided  through  Uninterruptable  Power  Supplies  (UPS) .  The 
UPS  conditions  the  power  (removes  line  spikes,  smoothes  power  fluctuations)  and 
provides  battery  backup  power  when  commercial  and  other  backup  power  is  not 
available. 

The  operational  positions  affected  by  the  PRM  are  Air  Traffic  (AT)  monitor 
controllers  and  Airways  Facilities  (AF)  field  technicians.  The  monitor  controllers 
interface  to  the  PRM  system  through  the  PRM  display  console.  The  field  technicians 
interface  with  the  PRM  through  the  PSD  and  the  PRM  display  consoles. 
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TRANSMIT/RECEIVE 

SITE 


OPERATIONS  SITE  - 
(EQUIPMENT  ROOM) 


OPERATIONS  SITE 
(TRACON) 


FIGURE  3.1-1.  PRM  SYSTEM  BLOCK  DIAGRAM 


3 . 2  INTERFACES  OVERVIEW . 


The  PRM  system's  only  equipment  interface  consist  of  a  passive,  read-only  interface 
with  the  ARTS  system.  Redundancy  is  achieved  by  connecting  each  of  two  PRM/ARTS 
interface  units  to  separate  Multiplexed  Display  Buffer  Memory  (MDBM)  ports  of  the 
ARTS.  The  interface  captures  ARTS  B  and  C  words  to  correlate  aircraft 
identification  and  type  data  to  PRM  tracks  for  display  on  the  PRM  display  consoles. 

4.  TEST  PROGRAM  DESCRIPTION. 

4.1  APPROACH  AND  CONCEPT. 

The  concept  of  the  LP  PRM  OT&E  test  effort  is  to  ensure  that  the  PRM  system  complies 
with  the  Critical  Operational  Issues  (COI)  imposed  by  the  LP  PRM  Master  Test  Plan. 

The  overall  approach  of  the  test  effort  is  to  separately  evaluate  the  operational 
suitability  and  operational  effectiveness  of  the  PRM  system  for  these  COIs  and  for 
additional  OT&E  test  components  derived  from  FAA  Order  1810. 4B.  Operational 
Effectiveness  centers  around  the  technical  performance  of  the  PRM  System,  while 
Operational  Suitability  centers  on  the  usability  of  the  PRM  System. 

Separate  Measures  of  Suitability  (MOS)  and  Measures  of  Effectiveness  (MOE)  have  been 
developed  for  each  COI.  Likewise,  each  MOE  has  been  further  factored  into  Measures 
of  Performance  (MOP)  and  finally  to  Minimiim  Acceptable  Operational  Requirements 
(MAORs) .  These  resultant  MOS  and  MAORs  are  the  critical  success  criteria  that  the 
PRM  system  must  meet  to  successfully  pass  the  OT&E  evaluation.  The  COIs,  along  with 
their  corresponding  Measures  and  MAORs,  are  listed  in  table  4.1-1. 

4.1.1  Evaluation  Approach. 

The  approach  for  evaluating  the  Operational  Effectiveness  of  the  PRM  is  to  conduct  a 
review  of  the  contractor  performed  Development  Test  and  Evaluation  (DT&E)  and  Site 
Acceptance  Tests.  This  review  will  evaluate  whether  each  of  the  MOEs  had  been 
satisfactorily  tested  and  whether  the  results  meet  the  MAORs. 

The  approach  for  evaluating  the  Operational  Suitability  of  the  PRM  is  to  expose  the 
test  participants  (AT  controllers  and  AF  technicians)  to  the  PRM  system  in  its 
operational  environment  while  they  perform  specified  operational  procedures.  The 
test  participants  will  then  fill  out  questionnaires  using  their  experience  and 
judgment.  The  questionnaires  will  be  evaluated  by  the  Test  Director  along  with  the 
test  participants  to  obtain  a  consensus  of  opinion  for  each  MOS. 

4.1.2  Critical  Operational  Issues  (COI) /Test  Requirement  Summary. 

The  COIs  of  the  PRM  and  the  method  to  be  used  to  measure  the  suitability  and 
effectiveness  of  the  PRM  system  in  verifying  each  COI  is  listed  in  table  4.1-1. 

These  COIs  were  developed  from  the  TVRTM  of  the  LP  PRM  Master  Test  Plan. 
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TABLE  4.1-1.  CRITICAL  OPERATIONAL  ISSUE  TEST  MATRIX. 


Critical  Operational 
Issue 

Measures  of  Suitability 

Measures  of 
Effectiveness 

Measures  of 
Performance 

Minimum  Acceptable 
Operational  Requirement 

1.  Coverage  Area: 

The  area  that  the  system 
acquires/  tracks  and 
displays  aircraft. 

The  coverage  area 
presented  on  the  PRM 
display  consoles  is 
sufficient  for  PRM 
monitoring  operations  for 
all  applicable  approaches. 

Elevation 

Coverage . 

Minimum  Elevation 
Coverage. 

Maximum  Elevation 
Coverage . 

-2deg  for  range  500ft- 
3nmi 

1.5deg  for  range  >3nmi 

Top  of  Approved  Keyhole 

Range  Coverage. 

Minimum  Range 

Coverage. 

1000ft  with  accuracy 

500  feet  w/o  accuracy 

Maximum  Range 

Coverage. 

Extent  of  Approved 
Keyhole 

Azimuth  Coverage. 

Minimum  Azimuths 

Covered . 

Extent  of  Approved 
Keyhole 

2.  Display  Mapping:  The 
map  data  presented  on 
the  PRM  display 
consoles. 

The  map  data  presented  on 
the  PRM  display  consoles 
is  comprehensive  enough 
for  PRM  monitoring 
operations . 

Map  Accuracy. 

Map  Runway  Accuracy. 

+/-  system  rng  &  az 
accuracy 

Map  Obstructions 
Accuracy. 

+/-  system  rng  s  az 
accuracy 

Map  NTZ  Accuracy. 

+/-  system  rng  &  az 
accuracy 

Map  Final  Approach 
Accuracy. 

+/“  system  rng  &  az 
accuracy 

3.  Sensor  Accuracy; 

The  accuracy  with  which 
the  system  presents 
targets  on  the  PRM 
display  consoles  as 
correlated  to  tracked 
aircraft  position. 

None. 

Range  Accuracy. 

Mean  Range  Bias. 

+/-  30ft. 

Mean  Range  Jitter. 

30ft  std  dev 

Azimuth  Accuracy. 

RMS  Azimuth  Accuracy. 

+/-  2.0  mrad  for 
elevations  <10  deg 

4.  Display  Resolution: 

The  capability  to 
resolve  two  closely 
spaced  aircraft. 

Closely  spaced  targets  can 
be  resolved  on  the  PRM 
display  consoles 
sufficiently  enough  for 

PRM  monitoring  operations. 

Tracked  Target 
Resolution. 

Azimuth  Resolution. 

.57  deg  @98% 

Range  Resolution. 

600ft  8  98% 

5.  System  Capacity; 

The  number  of  aircraft 
the  PRM  system  can 
acquire,  track  and 
display. 

The  capacity  of  the  system 
is  sufficient  to  monitor 
all  foreseeable  aircraft 
in  the  coverage  areas. 

Maximum  track 
capacity,  for 
each  applicable 
update  rate. 

Maximum  displayed 
tracks. 

25  @  1  sec  (dual) 

35  @  2.4  sec  (triple) 

Maximum  ground  tracks. 

15  (dual) 

15  (triple) 

6.  ARTS  Interface; 

The  PRM/ARTS  Interface 
must  be  a  passive  (read¬ 
only,  one-way) 
interface. 

None. 

PRM/ARTS 

Interface  effects 
on  the  ARTS 
system. 

Effects  on  TIRTS  system 
due  to  PRM/ARTS 

Interface  operations. 

No  effect 
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Critical  Operational 
Issue 

Measures  of  Suitability 

Measures  of 
Effectiveness 

Measures  of 
Performance 

Minimum  Acceptable 
Operational  Requirement 

7.  System  Reliability; 

An  estimate  of  how 
frequently  the  PRM 
system  will  fail. 

None. 

Reliability. 

Mean  Time  Between 
Critical  Failures. 

2190  hours 

Availability. 

Availability. 

.  999 

Mean  Time  To  Repair. 

30  min  with  approved 
exceptions 

8.  System  Failures: 

The  capability  to  detect 
system  failures  and 
provide  visual  and  aural 
alarms. 

The  visual  and  aural 
alarms  provided  on  the  PRM 
Display  Console  for  system 
failures  are  suitable  for 
notifying  appropriate 
personnel. 

Detection  of 

System  Faults. 

Minimum  Percent  Fault 
Detection. 

95% 

The  visual  and  aural 
alarms  provided  on  the  PRM 
Status  Display  for  system 
failures  are  suitable  for 
notifying  appropriate 
personnel. 

False  Alarm  Rate. 

Maximum  False  Alarm 
Rate. 

No  false  alarms 

9.  Tracked  Target: 

Display  of  tracked 
targets,  ARTS  data 
correlation,  and 
timeliness  of  data. 

The  tracking  and  display 
of  aircraft  is  sufficient 
for  PRM  monitoring 
operations  for  all 
applicable  approaches. 

Probability  of 
Detection. 

Minimum  Probability  of 
Detection  for  discreet 
beacon  targets. 

99% 

Track  History 

All  Displayed  Tracks 
have  a  Track  History. 

100%  moving  targets 

Track  Projection 
Vector 

All  Displayed  Tracks 
have  a  Projection 

Vector. 

100%  moving  targets 

The  timeliness  of  the 
displayed  targets  is 
sufficient  for  PRM 
monitoring  operations. 

System 

Throughput . 

Maximum  time  from 
aircraft  reply  to 
target  display. 

0.5  sec 

The  correlation  and 
display  of  ARTS  data  is 
comprehensive  enough  for 

PRM  monitoring  operations. 

Correlation  of 
Data. 

Probability  of  ARTS 
correlation. 

100%  for  discreet 
targets  with  available 
ARTS  data 

ARTS  Data 
Integrity. 

Maximum  False  Data 

Rate. 

0% 

10.  Display  Console: 

The  PRM  display 
console *s  quality  and 
controls. 

The  PRM  display  is  of 
sufficient  quality  for  PRM 
monitoring  operations. 

None. 

None. 

N/A 

The  PRM  display  console 
contains  all  controls 
(character  size, 
intensity,  data  block 
offset,  etc)  sufficient 
for  PRM  monitoring 
operations. 

PRM  Display 

Console  Keyboard 
Commands . 

All  PRM  Display 

Console  Keyboard 
Commands  function 
accurately. 

meets  PRM  spec  and 
design 

The  contrast  and 
brightness  of  the  PRM 
display  console  can  be 
adjusted  suitably  for 

TRACON  lighting 
conditions. 

Brightness  and 
Contrast 

Controls . 

Availability  of 
Brightness  and 

Contrast  Controls. 

Front  of  PRM  Display 
Console 

Critical  Operational 
Issue 

Measures  of  Suitability 

Measures  of 
Effectiveness 

Measures  of 
Performance 

Minimum  Acceptable 
Operational  Requirement 

11.  Recording,  Playback 
and  Listing: 

The  capability  to 
record,  replay,  and  list 
target  information. 

The  playback  of  recorded 
data  suitably  simulates 
actual  PRM  data  and 
operations. 

Data  Recording 
Integrity. 

Maximum  False/Missing 
Data  Rate. 

0% 

Data  Playback 
Integrity. 

Maximum  False/Missing 
Data  Rate. 

0% 

The  listing  of  PRM  data 
supplies  appropriate 
operational  data. 

Data  Listing 
Integrity. 

Maximum  False/Missing 
Data  Rate. 

0% 

12.  False  Targets: 

The  capability  to 
inhibit  the  number  of 
false  targets  presented 
on  the  PRM  display 
consoles. 

The  number  of  false 
targets  displayed  on  the 

PRM  Display  Console  is  few 
enough  for  PRM  monitoring 
operations. 

False  Targets  due 
to  fixed 
reflectors. 

Maximum  Number  of 

False  Tracks. 

<5%  of  false  reports 
due  to  known  stationary 
reflectors 

Split  Tracks. 

Maximum  Time  Until 
Removal . 

5  updates 

13.  NTZ  Alerts: 

The  capability  to 
provide  alerts  when 
targets  are  projected  to 
enter  the  NTZ. 

The  Aural  NTZ  projection 
is  suitable  for  PRM 
monitoring  operations. 

NTZ  Projection 
Alerts  Integrity. 

Probability  of  Alert. 

100% 

Maximum  False  Alert 

Rate. 

0% 
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4.1.3  Minimum  Acceptable  Operational  Requirements  (MAOR) . 


The  MAORs  are  listed  in  table  4.1-1:  Critical  Operational  Issue  Test  Matrix. 

4.1.4  Activities  Leading  to  Test. 

Prior  to  the  commencement  of  these  tests  and  evaluations  the  following  activities 
must  be  accomplished: 

a.  The  LP  PRM  has  successfully  completed  the  contractor  performed  DT&E  and 
Site  Acceptance  Tests. 

b.  This  test  plan  and  subsequent  test  procedures  are  approved. 

c.  Necessary  coordination  between  ACW,  the  local  user  organizations  (AT  and 
AF) ,  and  the  Program  Office  has  been  completed. 

d.  The  test  participants  (AT  controllers  and  AF  technicians)  must  have 
completed  the  contractor  provided  training. 

e.  The  PRM  system  has  been  placed  under  formal  FAA  configuration  control. 

4 . 2  TEST  ENVIRONMENT . 

The  LP  PRM  OT&E  test  effort  will  be  conducted  at  an  operational  using  the  full 
operational  PRM/NAS  configuration. 

4.2.1  Test  Location. 

The  OT&E  on  the  first  article  LP  PRM  system  will  be  perfoirmed  at  Minneapolis-St . 

Paul  International  Airport. 

4.3  TEST  AND  ANALYSIS  TOOLS. 

There  are  no  special  test  and  analysis  tools  required. 

4.4  TEST  AND  EVALUATION  DESCRIPTIONS. 

The  LP  PRM  OT&E  test  effort  consists  of  two  main  test  phases  and  multiple  sub-phases 
as  discussed  below.  Further  descriptions  for  each  test  sub-phase  are  provided  in 
appendix  A.  Table  4.4-1:  LP  PRM  OT&E  Test  Planning  Matrix  lists  the  various  test 
phases  and  sub-phases  along  with  the  schedule  estimates  and  the  number  and  type  of 
required  test  participants. 

The  Operational  Effectiveness  Test  Phase  consists  of  a  review  of  the  contractor 
performed  DT&E  and  Site  Acceptance  Tests.  This  review  will  evaluate  whether  each  of 
the  MOE  had  been  satisfactorily  tested  and  whether  the  results  meet  the  MAORs.  This 
review  will  be  conducted  solely  by  test  engineers  and  does  not  require  the  PRM 
system.  The  expected  duration  of  this  review  is  2  weeks. 

The  Operational  Suitability  Test  Phases  will  expose  the  test  participants 
(AT  controllers  and  AF  technicians)  to  the  PRM  system  in  an  operational  environment 
while  they  perform  specified  operational  procedures.  These  tests  will  be  conducted 
in  two  separate  phases:  AT  Suitability  and  AF  Suitability.  Each  of  these  phases  is 
focused  on  the  specific  test  participants. 

The  AT  Suitability  Test  Phase  will  be  subdivided  into  four  tests:  a  PRM  Display 
Console  Test,  a  Flight  Test,  a  System  Failure  Test,  and  a  Data  Playback  and  Listing 
Test.  All  four  tests  will  require  four  AT  controllers.  The  Data  Playback  and 
Listing  Test  will  also  require  one  AT  supervisor.  The  estimated  duration  of  the  AT 
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Suitability  Test  Phase  is  1  week.  Estimated  duration  for  each  test  is  listed  in 
table  4.4-1:  LP  PRM  OT&E  Test  Planning  Matrix. 


The  AF  Suitability  Test  Phase  will  be  subdivided  into  three  tests:  Fault  Detection 
and  Isolation  Test,  LRU  Removal  and  Replacement  Test,  and  Maintenance  Procedures 
Test.  Each  test  will  require  two  AF  technicians.  The  estimated  duration  of  the  AF 
Suitability  Test  Phase  is  3  weeks.  Estimated  duration  for  each  test  is  listed  in 
table  4.4-1. 

TABLE  4.4-1.  LP  PRM  OT&E  TEST  PIANNING  MATRIX. 


OT&E 

Test  Phase 

Test  Title 

Test  #s 

Test 

Participants 

System 

Requirements 

Test 

Duration 

Operational 

Effectiveness 

DT&E  Data  Review 
E.l 

Test  Eng- 2 

DT&E  Test  Procs 
Test  Reports 

2  Weeks 

AT  Suitability 

PRM  Display  Console 
Test 

ATS .  1 

Test  Eng-2 

AT  Ctlrs-4 

Full  PRM  System 

1  Day 

AT  Suitability 

Flight  Test 

ATS.  2 

Test  Eng-2 

AT  Ctlrs-4 

Full  PRM  System 
Test  Aircraft-2 

2  Days 

AT  Suitability 

System  Failure  Test 
ATS.  3 

Test  Eng-2 
■AT  Ctlrs-4 

Full  PRM  System 

1  Day 

AT  Suitability 

Data  Playback  & 
Listing  Test 

ATS.  4 

Test  Eng-2 

AT  Ctlrs-4 

AT  Sup-1 

Full  PRM  System 

1  Day 

AF  Suitability 

Fault  Detection  & 
Isolation  Test 
AFS.  1 

Test  Eng-2 

AF  Techs-2 

Full  PRM  System 

2  Days 

AF  Suitability 

LRU  Removal  & 
Replacement  Test 
AFS .  2 

Test  Eng-2 

AF  Techs-2 

Full  PRM  System 

10  Days 

AF  Suitability 

Maintenance 
Procedures  Test 
AFS.  3 

Test  Eng-2 

AF  Techs-2 

Full  PRM  System 

3  Days 

5  .  TEST  MANAGEMENT  . 

5.1  TEST  MANAGEMENT  ORGANIZATION. 

ACW-IOOA  has  the  primary  responsibility  for  the  direction  of  test  planning,  test 
conduct,  and  test  reporting  activities  associated  with  the  LP  PRM  OT&E  program.  The 
roles  and  responsibilities  of  other  organizations  involved  in  the  LP  PRM  OT&E  test 
effort  are  outlined  below. 

a.  Roles  and  Responsibilities. 

Organizational  roles  and  responsibilities  will  be  as  described  in  FAA  Order 
1810. 4B. 
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b.  Other  Participating  Organizations. 


Other  organizations,  not  covered  by  FAA  Order  1810. 4B,  that  will  take  part  in 
the  LP  PRM  OT&E  effort.  Their  roles  and  responsibilities  are  listed  below: 

AGL-’400  Great  Lakes  Region,  Airway  Facilities  Division  and  the  Minneapolis- 
St.  Paul  International  Airport  Airways  Facilities  Sector  Office  (AFSO) :  Assist  with 
the  Site  Acceptance  equipment  inspections,  i.e.,  power/safety,  provide  AF 
technicians  for  AF  Suitability  tests,  and  review  AF  Suitability  test  procedures. 

AGL-500  Great  Lakes  Region,  Air  Traffic  Division  and  Minneapolis-St .  Paul 
International  Airport  Air  Traffic  Control  Tower  (ATCT) :  Assist  in  flight  test 
planning,  provide  AT  controllers  for  the  AT  Suitability  tests,  and  review  AT 
Suitability  test  procedures. 

c.  Test  Conduct  Teams. 


The  test  conduct  teams  shall  consist  of  two  Test  Engineers  and  the  Test 
Participants,  i.e.,  AT  controllers  or  AF  technicians.  One  of  the  Test  Engineers 
will  be  designated  as  Test  Lead,  the  other  as  Test  Conductor. 

The  Test  Lead  will  be  responsible  for  briefing  the  Test  Participants  about  the 
test,  verifying  that  the  Pretest  conditions  have  been  met,  filling  out  the  test 
mission  log,  and  deciding  when  the  test  has  been  completed. 

The  Test  Conductor  shall  work  with  the  Test  Participants  to  assist  and  guide 
them  through  the  test  conduct  and  questionnaire  process. 

5.2  TRAINING. 

a.  Test  Developer  Training. 

The  FAA  test  developers  will  be  required  to  be  familiar  with  the  complete  PRM 
system  including:  the  PRM  specification,  the  various  system  configurations,  and  the 
PRM/NAS  interfaces.  This  knowledge  will  have  been  gained  through  participation  in 
the  contractor  DT&E  test  efforts  of  the  Demonstration  PRM,  the  Upgrade  PRM,  and  the 
LP  PRM  programs.  The  test  developers  will  also  be  required  to  know  the  operational 
use  of  the  PRM  including:  AT  Operations,  AF  Operations,  and  the  operational 
requirements  of  the  PRM.  This  knowledge  will  have  been  acquired  through 
participation  in  the  PRM  Demonstration  Flight  Tests,  the  Upgrade  PRM  OT&E  test 
effort,  as  well  as  the  preparation  for  this  OT&E  test  effort. 

b.  Test  Participant  Training. 

The  test  participants  who  take  part  in  the  OT&E  test  effort  will  be  either 
qualified  AT  controllers  or  AF  technicians.  Both  the  AT  controllers  and  the  AF 
technicians  will  be  given  formal  training  on  the  PRM  system  prior  to  their 
involvement  in  the  PRM  OT&E  test  effort. 

c.  Special  Training  Requirements. 

There  are  no  special  training  requirements. 

5.3  SYSTEM  CONFIGURATION  MANAGEMENT. 

Prior  to  the  start  of  the  OT&E  effort,  the  PRM  system  will  be  placed  under  FAA 
configuration  control.  The  only  changes  made  to  the  system  will  require  PRM  Program 
Office  approval.  Additionally,  an  inspection  of  the  PRM*s  maintenance  log  will  be 
made  before  each  OT&E  test  to  ascertain  whether  any  maintenance  actions  were  made  to 
the  system. 
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5.4  TEST  READINESS  CRITERIA. 


Before  this  OT&E  effort  is  started,  the  PRM  system  should  be  accepted  in  its  final 
form;  e.g.,  hardware,  software,  operational  documentation,  and  training  courses. 

The  test  participants  should  have  received  the  contractor  provided  training. 

Nonfinal  items  in  these  areas  could  result  in  changes  to  the  MOE  and  MOS  collected 
in  this  test  effort. 

5.5  TEST  EXECUTION. 

The  actual  process  for  conducting  a  test  session  is  divided  into  three  distinct 
phases.  A  pretest  review,  the  actual  test  conduct  and  a  post-test  review.  The  test 
mission  log  will  be  used  to  document  the  activities  of  all  three  of  these  phases. 

The  pretest  section  of  the  mission  log  shall  be  used  to  document  any  open  items  (to 
be  determined  or  discrepancies  from  the  test  procedure) ,  a  list  of  the  test  conduct 
members,  and  any  discrepancies  from  the  planned  test  configuration.  The  test 
conduct  section  will  be  used  to  record  specific  test  events  as  noted  in  the  test 
procedure.  The  post-test  section  shall  be  used  to  document  any  deviations  from  the 
test  procedure  that  were  not  noted  in  the  pretest  section,  a  summary  of  the  test 
conduct,  and  a  preliminary  assessment  of  the  test  results.  These  phases  are 
discussed  in  the  following  subsections. 

5.5.1  Pretest  Review. 


This  review  will  be  held  prior  to  the  start  of  each  test  session.  The  purpose  of 
this  review  will  be  to  accomplish  the  following: 

a.  Review  the  actual  hardware  and  software  configurations  of  the  test 
environment  against  that  planned  in  the  test  procedure.  If  the  actual  configuration 
is  different  from  that  planned  the  Test  Lead  will  decide  whether  the  test's 
objectives  can  still  be  met  or  whether  to  postpone  the  test.  Identify  any  needed 
changes  to  the  test  procedure  and  redline  accordingly. 

b.  The  Test  Lead  will  brief  the  Test  Participants  on  the  objectives  of  the 
test  and  describe  the  test  execution  of  the  test. 

c.  Fill  out  the  pretest  section  of  the  Test  Mission  Log. 

5.5.2  Test  Conduct. 

During  the  test  conduct  the  Test  Conductor  will  guide  and  assist  the  Test 
Participants.  The  Test  Lead  will  fill  out  the  test  conduct  section  of  the  test 
mission  log. 

5.5.3  Post-Test  Review. 

This  review  will  be  held  upon  the  completion  of  each  test.  The  purpose  of  this 
review  will  be  to  accomplish  the  following: 

a.  Identify  any  procedure  changes  or  test  discrepancies  and  their 
significance  to  the  test  results. 

b.  Identify  the  ”as-run”  hardware  and  software  configurations. 

c.  Summarize  the  actual  test  conduct  and  determine  a  preliminary  assessment 
of  the  test  results  and  system  deficiencies. 

d.  Fill  out  the  post-test  section  of  the  Test  Mission  Log. 
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5.6  TEST  COMPLETION  CRITERIA. 


The  initial  LP  PRM  OT&E  effort  will  be  complete  when  a  determination  has  been  made 
for  each  MOE  and  MOS  listed  in  this  plan.  The  complete  OT&E  effort  will  be  complete 
when  any  critical  system  deficiencies  have  been  resolved  and  the  corresponding  MOE 
and  MOS  have  been  reevaluated. 

The  DT&E  Data  Review  shall  be  complete  when  a  determination  has  been  made  for  each 
requirement.  This  determination  shall  include  whether  the  DT&E  test  conducted  was 
sufficient  and  whether  the  results  meet  the  MAORs . 

The  AT  and  AF  Suitability  test  phases  will  be  complete  when  a  final  consensus  has 
been  determined  for  each  MOS. 

5.7  TEST  REPORTS. 

A  Quick  Look  Report  will  be  provided  to  the  PRM  Program  Manager  within  2  weeks 
following  the  completion  of  the  OT&E  tests.  The  Quick  Look  Report  will  highlight 
any  system  deficiencies  found  during  the  OT&E  effort.  For  each  deficiency,  the 
Quick  Look  Report  will  provide  a  description,  a  risk  assessment,  and  a  proposed 
resolution.  The  classifications  for  risk  assessment  are  defined  as; 

High  Risk  “  The  deficiency  constitutes  a  risk  to  the  deployment  of  further  LP  PRM 
systems . 

Moderate  Risk  -  The  deficiency  constitutes  a  risk  to  the  commissioning  of  the  LP  PRM 
system. 

Low  Risk  *“  The  deficiency  constitutes  a  risk  to  the  operational  use  of  the  LP  PRM 
system  that  can  be  resolved  through  post-commissioning  modifications. 

A  Final  Test  Report  will  be  provided  to  the  PRM  Program  Manager  within  2  months 
following  the  completion  of  the  OT&E  tests.  The  Final  Report  will  document  the 
results  of  the  OT&E  effort  and  highlight  the  technical  performance  of  the  PRM 
system. 

5.8  SYSTEM  DEFICIENCY  REPORTS. 

System  Deficiency  Reports  will  be  collected  throughout  the  OT&E  test  effort.  These 
reports  will  be  filled  out  for  all  system  deficiencies.  They  will  contain  a 
description  of  the  deficiency,  the  test  that  initiated  the  deficiency,  a  risk 
assessment,  and  a  proposed  resolution. 

5.9  TEST  SCHEDULE. 

The  schedule  of  the  LP  PRM  OT&E  activities  is  depicted  in  table  5.9-1,  LP  PRM  OT&E 
SCHEDULE.  The  start  date  of  the  OT&E  effort  is  contingent  on  the  acceptance  of  the 
LP  PRM  system  and  the  availability  of  the  required  test  participants. 
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TABLE  5.9-1.  LP  PRM  OT&E  SCHEDULE. 


Test 

Schedule  j 

Operational  Effectiveness 

DT&E  Review 

Weekl 

Week2 

Operational  Suitability 

AT  Suitability 

PRM  Display  Console  Test 

Flight  Test 

System  Failure  Test 

Data  Playback  &  Listing  Test 

Weekl  .  , 

TW' 

T 

F 

AF  Suitability 

Fault  Detection  &  Isolation  Test 

LRU  Removal  &  Replacement  Test 
Maintenance  Procedures 

Week2 

MT 

WTF 

Weeks  .. 

MTWTF 

Week4 

MT 

WTF 

5.9.1  Planning  Considerations  and  Limitations. 

The  major  planning  consideration  and  limitation  of  this  OT&E  effort  deals  with  the 
limitation  of  test  aircraft.  This  effort  calls  for  two  test  aircraft  primarily  to 
demonstrate  nontransgression  zone  (NTZ)  blunders.  These  types  of  flights  can  only 
be  performed  during  VFR  weather  conditions.  While  the  schedule  shows  specific  days 
for  the  flight  tests,  the  aircraft  will  be  scheduled  for  the  full  week  of  AT 
Suitability  Tests.  The  schedule  also  contains  2  days  for  the  flight  test  although 
only  1  day  of  flight  tests  is  required.  Having  only  two  flight  aircraft  will  also 
preclude  the  demonstration  of  overload  conditions  on  the  PRM  system.  These  events 
will  be  covered  in  the  contractor  provided  training  and  thus  AT  controller  opinions 
relating  to  capacity  related  MOS  will  still  be  collected. 

The  effectiveness  of  the  PRM/ Automated  Radar  Terminal  System  (ARTS)  interface  is 
being  conducted  as  part  of  the  DT&E  Data  Review.  This  can  be  done  since  the 
interface  will  be  extensively  tested  on  site  as  part  of  the  contractor's  Site 
Acceptance  Tests . 

Note  that  as  part  the  Upgrade  PRM's  OT&E  test  a  test  was  conducted  to  ensure  that 
the  addition  of  the  PRM  into  the  Raleigh-Durham  air  traffic  environment  did  not 
degrade  the  performance  of  the  current  beacon  interrogator.  This  test  is  not  being 
conducted  as  part  of  this  OT&E  effort  since  the  LP  PRM  has  the  same  output  power  and 
employs  the  same  interrogation  scheme  and  pattern  as  the  Upgrade  PRM. 

5.10  PERSONNEL  RESOURCE  REQUIREMENTS. 

The  personnel  required  for  the  LP  PRM  OT&E  effort  include  Test  Engineers, 

AT  controllers,  AT  supervisors  and  AF  technicians. 

The  Test  Engineers  will  be  supplied  by  the  FAA  Technical  Center,  ACW-100.  They  will 
be  required  to  be  familiar  with  the  complete  PRM  system  including:  the  PRM 
specification,  the  various  system  configurations,  and  the  PRM/NAS  interfaces.  The 
test  developers  will  also  be  required  to  be  familiar  with  the  operational  use  of  the 
PRM  including:  AT  and  AF  Operations,  and  the  operational  issues  addressed  by  this 
OT&E  effort. 

The  AT  controllers  and  the  AT  supervisor  will  be  supplied  by  the  Minneapolis-St  Paul 
International  Airport  ATCT.  They  will  need  to  have  completed  the  contractor 
provided  training  on  the  PRM  system.  The  AT  controller's  role  in  the  OT&E  effort 
will  be  to  determine  if  the  PRM  system  meets  the  MOS  for  the  AT-related  COIs,  The 
AT  supervisor's  role  will  be  to  determine  if  the  PRM  system  meets  the  MOS  for  the  AT 
Management  related  COIs,  e.g.,  data  recording,  playba'ck  and  listing. 
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The  AF  technicians  will  be  supplied  by  the  Minneapolis-St .  Paul  International 
Airport  AFSO.  They  will  need  to  have  completed  the  contractor  provided  training  on 
the  PRM  system.  The  AF  technician's  role  in  the  OT&E  effort  will  be  to  determine  if 
the  PRM  system  meets  the  MOS  for  the  AF-related  Operational  Issues. 

The  number  and  type  of  test  participants,  which  tests  they  are  required  for,  and 
when  those  tests  are  expected  to  be  conducted  is  listed  in  table  4.4-1,  LP  PRM  OT&E 
Test  Planning  Matrix  and  table  5.9-1,  LP  PRM  OT&E  Schedule. 
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6.  ACRONYMS  AND  GLOSSARY. 


AF 

AFSO 

ARTS 

AT 

ATCT 

BRS 

COI 

CPMS 

CS 

DT&E 

FAA 

IFR 

LP  PRM 

LRU 

MAOR 

MDBM 

MOE 

MOP 

MOS 

NAS 

NTZ 

OT&E 

PRM 

PSD 

RDS 

RPS 

TVRTM 

UPS 

VFR 


Airways  Facilties 
Airways  Facilties  Sector  Office 
Automated  Radar  Terminal  System 
Air  Traffic 

Air  Traffic  Control  Tower 
Beacon  Radar  Subsystem 
Critical  Operational  Issue 

Confidence  and  Performance  Monitoring  Subsystem 
Communication  Subsystem 
Developmental  Test  and  Evaluation 
Federal  Aviation  Administration 
Instrument  Flight  Rules 

Limited  Production  Precision  Runway  Monitor 
Line  Replaceable  Unit 

Minimum  Acceptable  Operational  Requirement 

Multiplexed  Display  Buffer  Memory 

Measure  of  Effectiveness 

Measure  of  Performance 

Measure  of  Suitability 

National  Airspace  System 

Non-Transgression  Zone 

Operational  Test  and  Evaluation 

Precision  Runway  Monitor 

PRM  Status  Display 

Radar  Display  Subsystem 

Recording  and  Playback  Subsystem 

Test  Verification  Requirements  Traceability  Matrix 
Uninterruptable  Power  Supply 
Visual  Flight  Rules 
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APPENDIX  A 


TEST  AND  EVALUATION  DESCRIPTIONS 


This  appendix  contains  a  brief  outline  for  each  test  planned  for  the  LP  PRM  OT&E 
effort.  Note:  the  Critical  Operational  Issues  (COI)  and  the  associated  Measures  of 
Suitability  (MOS),  Measures  of  Effectiveness  (MOE) ^  Measures  of  Performance  (MOP), 
and  Minimum  Acceptable  Operational  Requirements  (MAOR)  that  these  tests  address  are 
listed  in  table  4.4“1  of  section  4.1.  Additional  objectives  have  been  added  to  each 
test  derived  from  the  OT&E  test  components  of  FAA  Order-1810 . 4B.  The  system  and 
test  participant  requirements  are  listed  in  table  4.4-1  of  section  4.4.  The 
proposed  schedule  is  listed  in  table  5.9-1  of  section  5.9. 


A-1 


Test  Number  E-1 

Test  Phase  Operational  Effectiveness 

Test  Title  DT&E  Data  Review 

Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Effectiveness  and  Measures  of  Performance  for  each  of  the 
Critical  Operational  Issues  listed  in  table  4.1-1  of  section  4.1 
and  to  collect  the  Measures  of  Effectiveness  for  each  of  the 
OT&E  test  components  listed  below. 

Technical  Performance:  The  capability  of  the  PRM  System  to  meet 
specified  requirements. 

Maintainability:  The  capability  of  the  PRM  system  to  be 
maintained  at  the  Line  Replaceable  Unit  (LRU)  level. 

System  Training:  The  contractor  provided  training  for  PRM 
operation  and  maintenance. 

Documentation:  The  contractor  provided  documentation  for  PRM 
operation  and  maintenance. 

Supportability :  The  sparing  program  that  supports  the  PRM 
system. 

Safety:  The  capability  of  the  PRM  system  to  be  maintained  and 
operated  safely. 

Test  Criteria  For  the  Critical  Operational  Issues  covered  by  this  test  the 

success  criteria  is  to  verify  that  each  requirement  was 
satisfactorily  tested  and  the  results  meet  the  MAORs  from  table 
4.4-1  of  section  4.1.  For  the  additional  OT&E  test  components 
the  success  criteria  is  listed  below. 


OT&E  Test  Components 

Measures  of  Effectiveness 

Success  Criteria 

Technical  Performance 

PRM  Specification 

Meets  Each  Specified  Requirement 

Maintainability 

Fault  Isolation 

Passed  Fault  Isolation  DT&E  Tests 

Training 

Maintenance  Course  Content 

Received  Contractual  Approvals 

Operations  Course  Content 

Received  Contractual  Approvals 

Operational  Documentation 

Instruction  Book  Content 

Received  Contractual  Approvals 

Operator’s  Manual  Content 

Received  Contractual  Approvals 

Safety 

Safety  Related  Specifications 

Passed  DT&E  Safety  Inspections 

PRM  System  Design 

^Received  Contractual  Approvals 

Supportability 

Quantity  of  Spares 

Received  Contractual  Approvals 

Quality  of  Spares 

Each  Spare  has  been  Tested  Sufficiently  to 
Ensure  Proper  Operation  in  the  PRM  System. 

Test  Approach  The  DT&E  Data  Review  consists  of  a  review  of  the  contractor 

performed  DT&E  and  Site  Acceptance  Tests,  the  contractor  Spares 
Test  Program,  as  well  as  of  other  contract  related  activities, 
i.e.,  training  course  and  document  reviews  and  approvals.  This 
review  will  evaluate  whether  each  of  the  Measures  of  Effectiveness 


A-2 


Test  Conduct 


Execution  Time 


had  been  satisfactorily  tested  and  whether  the  results  meet  the 
corresponding  success  criteria.  This  review  will  be  conducted  by 
two  test  engineers  and  does  not  require  the  PRM  system. 

Before  the  DT&E  Data  Review  can  be  conducted  the  contractor 
performed  DT&E  and  Site  Acceptance  test  must  be  completed.  This 
review  will  require  copies  of  the  "as  run"  test  procedures  and  the 
associated  test  reports  (drafts  may  be  used) .  The  DT&E  Data 
Review  shall  be  complete  when  a  determination  has  been  made  for 
each  requirement.  This  determination  shall  include  whether  the 
DT&E  test  conducted  was  sufficient  and  whether  the  results  meet 
the  MAORs. 

The  expected  duration  of  this  review  is  2  weeks. 
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Test  Number 


ATS-1 


Test  Phase  AT  Suitability 

Test  Title  PRM  Display  Console  Test 


Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  Critical  Operational  Issues;  2  and  10  listed  in 
table  4 •1-1  of  section  4.1  and  to  collect  Measures  of  Suitability 
for  each  of  the  OT&E  test  components  listed  below. 

System  Training:  The  contractor  provided  training  for  Air  Traffic 
PRM  operation. 


Documentation:  The  contractor  provided  Operator's  Manual. 

Test  Criteria  The  success  criteria  for  each  of  the  Operational  Issues  evaluated 

in  this  test  is  to  meet  the  Measures  of  Suitability  as  presented 
in  table  4.1-11  of  section  4.1  and  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Training 

The  Training  provided  to  the  AT  controllers  must 
suitably  cover  the  commands,  controls,  and  features  of 
the  PRM  Display  Console. 

Operational  Documentation 

The  Operators  Manual  provided  to  the  AT  controllers 
must  suitably  cover  the  commands,  controls,  and 
features  of  the  PRM  Display  Console. 

Test  Approach  This  test  does  not  require  the  actual  control  of  aircraft,  but 

uses  what  ever  traffic  is  available  (targets  of  opportunity) 
simply  as  a  display  feature.  This  test  will  collect  opinions  from 
four  AT  controllers  (two  at  a  time)  by  having  them  use  the  PRM 
display  commands  and  controls  while  monitoring  targets  of 
opportunity  on  PRM  Display  Consoles.  The  AT  controllers  will  be 
given  a  set  of  questionnaires  that  will  lead  them  through  the  test 
by  having  them  enter  specific  commands  and  controls  and  answer 
corresponding  questions.  These  questions  investigate  the 
suitability  of  (1)  the  commands  and  controls  used,  (2)  the 
contractor  provided  documentation  and  training,  and  (3)  the  PRM 
display  presentation  and  map.  An  analysis  of  the  questionnaires 
will  be  conducted  by  the  test  lead  along  with  the  AT  controllers 
to  obtain  a  consensus  of  opinion  for  each  Measure  of  Suitability. 


Test  Conduct  This  test  requires  a  set  of  PRM  displays  in  their  normal 

operational  environment.  Associated  commands  and  controls  will  be 
grouped  together  for  each  specific  questionnaire.  The  test  can  be 
stopped  or  paused  between  questionnaires.  The  test  will  be 
complete  when  a  final  consensus  has  been  determined  for  each 
Measure  of  Suitability.  If  required,  portions  of  the  test  may  be 
rerun  to  obtain  this  consensus. 

Execution  Time  The  expected  duration  of  this  test  is  2  days. 
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Test  Nuinber 


ATS-2 


Test  Phase 


AT  Suitability 


Test  Title  Flight  Test 

Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  Critical  Operational  Issues:  1,  A,  5,  9,  12,  and 
13  listed  in  table  4.1-1  of  section  4.1  and  to  collect  Measures  of 
Suitability  for  each  of  the  OT&E  test  components  listed  below. 


System  Training:  The  contractor  provided  training  for  Air  Traffic 
PPM  operation. 

Documentation;  The  contractor  provided  Operator’s  Manual. 


Test  Criteria  The  success  criteria  for  each  of  the  Operational  Issues  evaluated 

in  this  test  is  to  meet  the  Measures  of  Suitability  as  presented 
in  table  4.1-1  of  section  4.1  and  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Training 

The  Training  provided  to  the  AT  controllers  must 
suitably  cover  the  required  system  knowledge  for 
performing  PPM  monitoring  operations . 

Operational  Documentation 

The  Operators  Manual  provided  to  the  AT  controllers 
must  suitably  cover  the  required  system  knowledge  for 
performing  PPM  monitoring  operations , 

Test  Approach  This  test  uses  two  FAA  test  aircraft  to  supply  targets  for  four  AT 

controllers  (two  at  a  time)  to  monitor  and  control  at  the  PPM 
Display  Consoles.  The  test  aircraft  will  fly  side  by  side 
simultaneous  approaches  that  will  include  as  a  minimum: 

NTZ  blunders 

radio  failure  transponder  alert 
missed  approach 

Additional  approaches  will  be  selected  by  the  Test  Participants 
(AT  controllers) .  Questionnaires  will  be  collected  from  the  AT 
controllers  to  obtain  their  opinions  on  the  Measures  of 
Suitability  for  each  Operational  Issue.  An  analysis  of  the 
questionnaires  will  be  conducted  by  the  test  director  along  with 
the  AT  controllers  to  obtain  a  consensus  of  opinion  for  each 
Measure  of  Suitability. 


Test  Conduct  This  test  requires  a  set  of  PPM  displays  in  their  normal 

operational  environment  including  operational  communications .  The 
test  will  be  complete  when  a  final  consensus  has  been  determined 
for  each  Measure  of  Suitability.  To  obtain  this  consensus 
portions  of  the  test  may  be  rerun  using  targets  of  opportunity. 

All  flights  should  be  conducted  under  visual  flight  rules  (VFR) . 

Execution  Time  The  expected  duration  of  this  test  is  1  day  with  a  second  day 
scheduled  in  case  of  weather  or  other  problems. 
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Test  Nvimber 


ATS-3 


Test  Phase  AT  Suitability 

Test  Title  System  Failures  Test 


Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  Critical  Operational  Issue  8  listed  in  table  4.1-1 
of  section  4.1  and  to  collect  Measures  of  Suitability  for  each  of 
the  OT&E  test  components  listed  below. 


System  Training:  The  contractor  provided  training  for  Air  Traffic 
PRM  operation. 


Documentation:  The  contractor  provided  Operator *s  Manual. 

Test  Criteria  The  success  criteria  for  each  of  the  Operational  Issues  evaluated 
in  this  test  is  to  meet  the  Measures  of  Suitability  as  presented 
in  table  4.1-1  of  section  4.1  and  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Training 

The  Training  provided  to  the  AT  controllers  must 
suitably  cover  the  required  system  knowledge  for 
performing  PRM  monitoring  operations . 

Operational  Dociomentation 

The  Operators  Manual  provided  to  the  AT  controllers 
must  suitably  cover  the  required  system  knowledge  for 
performing  PRM  monitoring  operations. 

Test  Approach  This  test  does  not  require  the  actual  control  of  aircraft,  but 
uses  targets  of  opportunity  simply  as  a  display  feature.  This 
test  will  collect  opinions  from  four  AT  controllers  {two  at  a 
time)  by  having  them  monitor  the  PRM  displays  as  PRM  system 
failures  are  demonstrated.  These  system  failures  will  include  as 
a  minimum: 


full  data  recording  tape  (single  and  dual) 
dectalker  failure  (single  and  dual) 
antenna  failure 

channel  failure  with  automatic  switch 
ARTS  Interface  failure  (single  and  dual) 

Additional  system  failures  will  be  selected  by  the  Test 
Participants  (AT  controllers) .  Questionnaires  will  be  collected 
from  the  AT  controllers  to  obtain  their  opinions  on  the  Measures 
of  Suitability  for  each  Operational  Issue.  An  analysis  of  the 
Questionnaires  will  be  conducted  by  the  test  director  along  with 
the  AT  Controllers  to  obtain  a  consensus  of  opinion  for  each 
Measure  of  Suitability. 


Test  Conduct  This  test  requires  a  fully  operational  PRM  system  in  its  normal 

operational  environment.  The  test  may  be  stopped  or  paused 
between  any  system  failure.  Any  system  failure  can  be  repeated  as 
necessary.  The  test  will  be  complete  when  a  final  consensus  has 
been  determined  for  each  Measure  of  Suitability. 

Execution  Time  The  expected  duration  of  this  test  is  1  day. 
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Test  Number  ATS- 4 

Test  Phase  AT  Suitability 

Test  Title  Data  Playback  and  Listing  Test 

Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  Critical  Operational  Issue  11  listed  in  table  4.1- 
1  of  section  4.1  and  to  collect  Measures  of  Suitability  for  each 
of  the  OT&E  test  components  listed  below. 

System  Training:  The  contractor  provided  training  for  Air  Traffic 
PPM  operation. 

Documentation:  The  contractor  provided  Operator *s  Manual. 

Test  Criteria  The  success  criteria  for  each  of  the  Operational  Issues  evaluated 

in  this  test  is  to  meet  the  Measures  of  Suitability  as  presented 
in  table  4.1-1  of  section  4.1  and  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Training 

The  Training  provided  to  the  AT  controllers  must 
suitably  cover  the  operation  of  the  data  recording, 
playback,  and  listing  functions  of  the  PRM  system. 

Operational  Documentation 

The  Operators  Manual  provided  to  the  AT  controllers 
must  suitably  cover  the  operation  of  the  data 
recording,  playback,  and  listing  functions  of  the  PRM 
system. 

Test  Approach  This  test  does  not  require  the  actual  control  of  aircraft,  but 

uses  targets  of  opportunity  and  the  data  tapes  collected  during 
test:  ATS. 2  Flight  Test.  This  test  will  use  four  AT  controllers 
and  one  AT  supervisor  to  record,  playback,  and  list  targets  of 
opportunity  and  flight  test  data.  Questionnaires  will  be 
collected  from  the  AT  controllers  and  the  AT  supervisor  to  obtain 
their  opinions  on  the  Measures  of  Suitability  for  each  Operational 
Issue.  An  analysis  of  the  questionnaires  will  be  conducted  by  the 
test  director  along  with  the  AT  controllers  to  obtain  a  consensus 
of  opinion  for  each  Measure  of  Suitability. 

Test  Conduct  This  test  requires  a  fully  operational  PRM  system  in  its  normal 

operational  environment  and  the  data  tapes  collected  during  test: 
ATS. 2  Flight  Test.  The  test  may  be  stopped  or  paused  any  time 
after  the  targets  of  opportunity  data  have  been  collected.  The 
test  will  be  complete  when  a  final  consensus  has  been  determined 
for  each  Measure  of  Suitability. 

Execution  Time  The  expected  duration  of  this  test  is  one  day. 
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Test  Number 


AFS-1 


Test  Phase  AF  Suitability 

Test  Title  Fault  Detection  and  Isolation  Test 

Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  each  of  the  OT&E  test  components  listed  below. 

Maintainability:  The  capability  of  the  PRM  system  to  be  maintained 
at  the  LRU  level. 


System  Training:  The  contractor  provided  training  for  PRM 
operation  and  maintenance. 

Documentation:  The  contractor  provided  documentation  for  PRM 
operation  and  maintenance. 

Safety:  The  capability  of  the  PRM  system  to  be  maintained  safely. 
Test  Criteria  The  success  criteria  is  listed  below. 


OT&E  Test  Components 


Maintainability 


Training 

Operational  Documentation 


Measures  of  Suitability 


The  PSD  must  be  suitably  usable  by  appropriate  trained  personnel  to 
detect  and  isolate  system  faults  to  the  Line  Replaceable  Unit  (LRU) 
level . 


The  Training  provided  to  the  AF  Technicians  must  suitably  cover  the 
operation  of  the  PSD  to  detect  and  isolate  faults. 


The  Instruction  Book  provided  to  the  AF  Technicians  must  suitably 
cover  the  operation  of  the  PSD  to  detect  and  isolate  faults. 


Test  Approach  This  test  will  collect  opinions  from  two  AF  technicians  by  having 
them  use  the  PSD  to  detect  and  isolate  faults  at  the  LRU  level 
placed  into  the  PRM  system  by  the  test  engineers .  As  a  minimum, 
the  LRUs  to  be  faulted  will  consist  of  those  listed  below: 


Test  Conduct 


Execution  Time 


Transmitter  Power  Supply 
Interrogator  Power  Amplifier 
ARTS  Interface  Unit 
Display  Console  Keyboard 
RFD  Power  Supply 
Antenna  Dipole  Column 


Data  Processor  CPU 
Receiver  Reply  Decoder 
Tape  Drive 

Display  Graphics  Driver 
RFD  Steering  Memory 
Ethernet  Switch 


Questionnaires  will  be  collected  from  the  AF  technicians  to  obtain 
their  opinions  on  the  Measures  of  S'uitability  for  each  Operational 
Issue.  An  analysis  of  the  questionnaires  will  be  conducted  by  the 
test  director  along  with  the  AF  technicians  to  obtain  a  consensus 
of  opinion  for  each  Measure  of  Suitability. 

This  test  requires  a  fully  operational  PRM  system  in  its  normal 
operational  environment.  The  test  may  be  stopped  or  paused  after 
each  fault  detection  and  isolation.  The  test  will  be  complete 
when  a  final  consensus  has  been  determined  for  each  Measure  of 
Suitability.  Specific  faults  may  be  tested  again  if  required. 

The  expected  duration  of  this  test  is  2  days. 
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Test  Number 


AFS-2 


Test  Phase  AF  Suitability 

Test  Title  LRU  Removal  and  Replacement  Test 

Test  Objectives  The  objective  of  this  test  is  to  collect  the  Measures  of 

Suitability  for  each  of  the  OT&E  test  components  listed  below. 

Maintainability:  The  capability  of  the  PRM  system  to  be  maintained 
at  the  LRU  level. 


System  Training:  The  contractor  provided  training  for  PRM 
operation  and  maintenance. 

Documentation:  The  contractor  provided  documentation  for  PRM 
operation  and  maintenance. 

Safety:  The  capability  of  the  PRM  system  to  be  maintained  safely. 

Supportability:  The  sparing  program  that  supports  PRM  maintenance 
operations . 


Test  Criteria  The  success  criteria  is  listed  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Maintainability 

The  design  of  the  PRM  system  must  allow  for  the 
removal  and  replacement  of  all  LRUs  of  the  PRM  system. 

Training 

The  Training  provided  to  the  AF  Technicians  must 
suitably  cover  the  removal  and  replacement  of  all  the 
LRUs  of  the  PRM  system. 

Operational  Documentation 

The  Instruction  Book  provided  to  the  AF  Technicians 
must  suitably  cover  the  removal  and  replacement  of  all 
LRUs  of  the  PRM  system. 

Safety 

The  design  of  the  PRM  system  must  allow  for  the  safe 
removal  and  replacement  of  all  LRUs  of  the  PRM  system. 

Supportability 

The  spares  provided  are  of  suitable  quantity  and 
quality  to  maintain  the  PRM  system. 

Test  Approach  This  test  will  collect  opinions  from  two  AF  technicians  by  having 

them  remove  and  replace  or  walk  through  the  removal  and 
replacement  sections  of  the  PRM  Instruction  Book  for  each  LRU  of 
the  PRM  system.  The  LRUs  that  will  be  removed  and  replaced  will 
consist  of: 

Frequency  Generator 
Cabinet  Blower 
Fiber  Optic  Transceiver 
Graphics  Driver 
RFD  Power  Supply 

Additional  LRUs  may  be  tested  depending  on  the  schedule.  These 
LRUs  will  be  selected  by  the  Test  Participants  (AF  technicians) . 
Questionnaires  will  be  collected  from  the  AF  technicians  to  obtain 


Receiver 
Tape  Drive 
Sony  CRT  Assembly 
Phase  Shifter 
Antenna  Dipole  Column 
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Test  Conduct 


Execution  Time 


their  opinions  on  the  Measures  of  Suitability  for  each  Operational 
Issue.  An  analysis  of  the  questionnaires  will  be  conducted  by  the 
test  director  along  with  the  AF  technicians  to  obtain  a  consensus 
of  opinion  for  each  Measure  of  Suitability. 

This  test  requires  a  fully  operational  PRM  system  in  its  normal 
operational  environment.  The  test  may  be  stopped  or  paused  after 
each  LRU  removal  and  replacement.  The  test  will  be  complete  when 
a  final  consensus  has  been  determined  for  each  Measure  of 
Suitability.  Specific  LRUs  may  be  tested  again  if  required. 

The  expected  duration  of  this  test  is  10  days. 
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Test  Number 


AFS-3 


Test 

Phase 

AF  Suitability 

Test 

Title 

Maintenance  Procedures  Test 

Test 

Objectives 

The  objective  of  this  test  is  to  collect  the  Measures  of 
Suitability  for  each  of  the  OT&E  test  components  listed  below 

Maintainability:  The  capability  of  the  PRM  system  to  be  maintained 
at  the  LRU  level. 

System  Training:  The  contractor  provided  training  for  PRM 
operation  and  maintenance. 

Documentation:  The  contractor  provided  documentation  for  PRM 
operation  and  maintenance. 

Safety:  The  capability  of  the  PRM  system  to  be  maintained  safely. 
Test  Criteria  The  success  criteria  is  listed  below. 


OT&E  Test  Components 

Measures  of  Suitability 

Maintainability 

Maintenance  Procedures  must  be  executable  by 
appropriate  trained  personnel 

Training 

The  Training  provided  to  the  AF  technicians  must 
suitably  cover  the  maintenance  procedures  of  the  PRM 
system. 

Operational  Documentation 

The  Instruction  Book  provided  to  the  AF  technicians 
must  suitably  cover  the  maintenance  procedures  of  the 

PRM  system. 

Safety 

The  design  of  the  PRM  system  must  allow  for  the  safe 
conduct  of  all  maintenance  procedures  of  the  PRM 
system. 

Test  Approach 


Test  Conduct 


This  test  will  collect  opinions  from  two  AF  technicians  by  having 
them  perform  each  of  the  maintenance  procedures  of  the  PRM  system. 

Questionnaires  will  be  collected  from  the  AF  technicians  to 
obtain  their  opinions  on  the  Measures  of  Suitability  for  each 
Operational  Issue.  An  analysis  of  the  questionnaires  will  be 
conducted  by  the  test  director  along  with  the  AF  Technicians  to 
obtain  a  consensus  of  opinion  for  each  Measure  of  Suitability. 

This  test  requires  a  fully  operational  PRM  system  in  its  normal 
operational  environment.  The  test  may  be  stopped  or  paused  after 
each  maintenance  procedure.  The  test  will  be  complete  when  a 
final  consensus  has  been  determined  for  each  Measure  of 
Suitability.  Specific  maintenance  procedures  may  be  retested  if 
required. 


Execution  Time  The  expected  duration  of  this  test  is  3  days. 
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APPENDIX  B 


TEST  VERIFICATION  REQUIREMENTS  TRACEABILITY  MATRIX 


The  TVRTM  below  shows  which  tests  collect  Measures  of  Effectiveness  and  Measures  of 
Suitability  for  each  of  the  Operational  Issues  evaluated  in  this  OT&E  effort. 


Operational  Issue 

Critical 

Tests  for 
Effectiveness 

Tests  for 
Suitability 

1. 

Coverage  Area 

Yes 

E-1 

ATS-2 

2. 

Display  Mapping 

Yes 

E-1 

ATS-1 

3. 

Sensor  Accuracy 

Yes 

E-1 

None 

4. 

Display  Resolution 

Yes 

E-1 

ATS-2 

5. 

System  Capacity 

Yes 

E-1 

ATS-2 

ARTS  Interface 

Yes 

E-1 

None 

7. 

System  Reliability 

Yes 

E-1 

None 

8. 

System  Failures 

Yes 

E-“l 

ATS-3 

9. 

Tracked  Targets 

Yes 

E-1 

ATS-2 

10. 

Display  Console 

Yes 

E-1 

ATS-1 

11.  Recording,  Listing  and 

Playback 

Yes 

E-1 

ATS-4 

12. 

False  Targets 

Yes 

E-1 

ATS-2 

13. 

NTZ  Alerts 

Yes 

E-1 

ATS-2 

14. 

Technical  Performance 

No 

E-1 

None 

15. 

Maintainability 

No 

E-1 

AFS-1,2,3 

16. 

Training 

No 

E-1 

ATS-1, 2, 3, 4 
AFS-1,2,3 

17. 

Operational  Documentation 

No 

'  E-1 

ATS-1, 2, 3, 4 
AFS-1,2,3 

18. 

Safety 

No 

E-1 

AFS-2,3 

19. 

Supportability 

No 

E-1 

AFS-2 

t 
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